{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boosting\n",
    "* 和bagging一样集成多个模型\n",
    "* 和bagging不一样, 每个模型都在尝试增强(Boosting)整体的效果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# AdaBoosting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "Ada Boosting 在每次学习后, 赋予误差大的点一个较大的权重值, 然后进行下一次学习, 按照这个过程循环"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJztnXuUVNWd77+/qq6GajKh5ZHRbvARh8HxgfaAjwzMTNRRYlRsMcFX7iQzGm7GJEbNBTHXwZblJCjrqjGPSYhmTFY0oRVs8bUwUZNcTMwFbECJMhpNpLs1AbTbSBd2dde+f5w61adO7b3PPo+qOlX1+6zFAupxzq5Tp35779/j+yMhBBiGYZjGIlHtATAMwzCVh40/wzBMA8LGn2EYpgFh488wDNOAsPFnGIZpQNj4MwzDNCBs/BmGYRoQNv4MwzANCBt/hmGYBqSp2gNQMW3aNHHkkUdWexgMwzA1xbZt2/YJIaZ7vS62xv/II4/E1q1bqz0MhmGYmoKI/mDyOnb7MAzDNCBs/BmGYRoQNv4MwzANCBt/hmGYBoSNP8MwTAPCxp9hGKYBiW2qJ9M49PT2Y82m3RgYzKCtNY1lC2ejs6O92sNimLqGjT9TVXp6+3HDhheQyY4BAPoHM7hhwwsAwBMAw5QRdvswVWXNpt0Fw2+TyY5hzabdVRoRwzQGbPyZqjIwmPH1OMMw0cDGn6kqba1pX48zDBMNbPyZqrJs4WykU8mix9KpJJYtnF2lETFMY8ABX6aq2EFdzvZhmMrCxp+pOp0d7WzsGabCsNuHYRimAWHjzzAM04Cw8WcYhmlA2PgzDMM0IGz8GYZhGhA2/gzDMA0IG3+GYZgGhI0/wzBMA8JFXgwTEdyXgKkl2PgzTARwXwKm1mC3D8NEAPclYGoNXvkzjIOgrhvuS8DUGpGs/Ino+0T0JyJ6UfH8R4loiIi25/+sjOK8DBMltuumfzADgXHXTU9vv+d7uS8BU2tE5fa5F8DHPF7zf4UQJ+X/rIrovAwTGWFcN9yXgKk1InH7CCF+SURHRnEshqkWYVw33JeAqTUq6fP/CBHtADAA4H8JIXZV8NwM40lbaxr9EkPf1po2igVwXwKmlqhUts/zAI4QQpwI4BsAemQvIqKlRLSViLbu3bu3QkNjGAuV6+b0Y6YHjgUwTFypiPEXQrwrhHgv/+/HAaSIaJrkdWuFEPOEEPOmT59eiaHVJT29/Zi/+mkcteIxzF/9NBspQzo72vG1xSegvTUNAtDemsbXFp+AZ17ey2mcTN1REbcPER0K4I9CCEFEp8CadPZX4tyNBhcbhcPtuunp7Ze6ggBO42Rqm0iMPxH9GMBHAUwjoj4ANwFIAYAQ4jsAPgHg34hoFEAGwCVCCBHFuZlidBkrbPz9YU+kKjiNk6llosr2udTj+W8C+GYU52L0cLFRdMgmUhtO42RqHZZ3qDO42Cg6dBPm1xafwDsppqZh419ncLFRMWGC36oJs701zYafqXnY+NcZqoyVRjRWYeQaAJ5ImfqGhd3qEC42sggb/I6yape1/pm4wcafqVuCBr+jNtScfsvEEXb7MDVBEN99kOB3WFeRDNb6Z+IIG38m9gQ1yKcfMx3keszLZ18OQ83pt0wcYePPxJ4gBrmntx/rt/XDWUlIAC6aq4+HqAxy/2AmsFQGp98ycYR9/g2IH592HAKVQVbOsglDAHjm5XHBQNlnUyl7AsF99csWzi7y+QOcNcRUH175Nxh+XCjl8H8HIcjK2WvCUH2204+ZXpLe6SSTHUPXxlI1cl1MgtNvmTjCK/86x726HR4ZNU5/DJsqGXbXYL+/fzADAopcOF4rZ502P6D+bD/+zR5ceupMPPPyXuUOYDCTRU9vf+GzmGTzcPotEzd45V/HyFa37wxnpa+VrZTDBCrD7hqc7wcsw28Hb01Wzl4FWqrPMCYE1m/rx7KFs9Gu2Vk44w2czcPUImz86xidMJkbmQslTKAyrEFU+ezbW9N4dsUZnqtoL1eL7jPY49TtLJyTh1eQmPsqMHGE3T41iolLxTSVUOVCCROoDJveGEV6pM7VIvts7vN0drTj5kd2SXdLzslD5WIioPA4F3YxcYNX/jWIqUtFtbolAIe0pDyDj2EClWHTG8udHml/tiS5KwEsJqdTmL/6abwznPWsFZC5mNwxCoBdQUy84JV/DWIaiFWtbgWAg9kc7rj4JCP3SZCVatj0xkqkR9qfy32eVIJwYGQUgxlrxW/HGwSAJFGREXdeH+dOjLt/MXGHjb8P4pDzDpi7ROyxfbl7B8ZcjdMy2TFcs257wbcd9ecwEUXTXc8oRdX8jnN4ZLTE1WNPAPZ17B/M4Np127H1D2/jls4TiiaBnt5+XLtue8nKH+DCLiY+UFy7Kc6bN09s3bq12sMo4E7nA6yVaDXytU+6+cnCqtSJHQx1c9SKx6SGyCbM53Aa8NaWFIQAhjJZqbG+secF/Pg3ezAmBIgsY5pzDEw7jp3dwFOrgKE+YPIM4MyVwJwlvsdrgtf1ckJAyQ5q/uqnlTEAk90Ww4SBiLYJIeZ5vY5X/obEpTduT28/DoyMljyeSpDSJaJzQwDW5/hy9w4A6mCkbJUOFLtMnKtld4Dzxp4X8KPn3ig8L4TaJ14yhp3dwCNXA9n8ZxjaY/0fKEwAUe7KvK6XEwGUjFm1MxMwC/bGZYfJ1Dcc8DUkLuJcazbtRnasdF36gYlN2swWXdUqYLkzdJW+yx7cURRgXvbgDnRt3KVNJXX6xn/8mz3a89tIr+dTq8YNv002Yz0O7wC4X0VQVQDXdMwq105rPoisG0dcqqqZ+oeNvyFxEedSTTaDw1mlkXNm7ehQZaPc/MiukgknOyakrifVeN0xBxXS6znUJ39x/nHdriyIMZVlOV1+2uHKCcA9ZtnkYQeRvSaoa9Zt54IxpiKw28eQuIhzqVwSrS2pEokBWUBSFrtwIptcVFXBpuMFrCwZrwlAeT0nz7BcPbLHod+VBXXXqbKc7nvuDU+ZCdMgstOo674T3WdkmKDwyt+QuIhzqWQLhIA0pfO+596Qioyp8tv97mR07iSnYbz01JnS17SkEt7X88yVQMo1rlTaelwz5rbWdKTuuls6T8AdF59kdA90drTj2RVn4PXV5+LZFWdgUCOrYVKJ7dWAhiuJGb/wyt8HcRDnUqVAXrtuu/T1dkDS/Z5LT52J9dv6jXYyremU0sUzMZXAhKYEhjJZpFMJZEZzEMJa6Tu182/pPAEACtk+SSJceurMwuNa7KweSbZPT28/hl0B8EWJzbg+1Y22g/vxx4nT8NWRT2JjbkHRa4K664LeAzqhOa+JSLfD5BaRTFAiSfUkou8DOA/An4QQx0ueJwBfB/BxAMMAPiOEeF53zLilesYdVXqhTTqVLDH0F81txzMv7y3J4JFl9Sx7YAeyOfm9Yh9LNpnIVsZRZbPIXFiLEptxa/PdSGOk8FhGNOP67JWFCSCKFF2/n0GXKmwrl8po9zi26ntXpf0y9U+lUz3vBfBNAD9UPH8OgFn5P6cC+M/830xE2Kt/mXm2q1KdZLJjeOblvUUGQrWK/NriE7DmkycqjZQthSwrJHPvOlpbUvjoyM+xLrEObRP2YWB4Gu586BIAV/k2xjJ3yfKm7iLDDwBpGsFXmh/AIwcXFE1o81c/HWgCCrLa9ipaC1pDEpcsNKb2iMT4CyF+SURHal5yAYAfCmub8RwRtRLRYUKIN6M4P2MZl61/eFsakDQN7qqCo10bd2H7TWejs6NdWQClCubahtE+7t8ffAb/kbobLWQZ6Bm0D6vEWtz2WBM6O26WHkO1ypYZuDbaJz3GodiH11efWzheGFdJVEFk21c/MJjB5HQKE1MJDA7LC+RUePUtYBgVlQr4tgNwpmv05R+rKvUWKFMFJFUpnm4DoUwjzTcvkb3HRhVAdu86ljd1Fwy/TQuN4MqRH0nf39Pbj2UPuGoMHtiBnt5+6VgGxDTpcezMICC83LROwjlIvwIB6xrbeksmktU2Xn0LGEZFpYy/zDKULBWJaCkRbSWirXv37pW8JTrqtZjGnWXS2dFubCB0q0XbMKqOdempM6WPu3cEqpV5W2K/dCLu2rirJNaQzQl0bdwlHcuduASjyYnFB3dkBgHhXSW662R6D0XVACYuWWhM7VGpbJ8+AM5cvxkABtwvEkKsBbAWsAK+5RxQXOQaKoGpSNqyhbNxjSJryDaMumPNO2JKyePuOMGAmIYZkgngTTFVqn2vyjIazGSlY1mw8Co0JU/U6gAFcZW4NYxSCZIGwJ33kC4oHKWvPg5ZaEztUSnjvxHAF4joJ7ACvUPV9vc3WqDMxECYNi9RHUv1uNO/ftvoEqx2+PwBIINm3JotFmkzXQXLz7lEK/qmkroeHhkt6s1r444RvDOcRTKhFnwYyLt/dHEF9tUz1SYStw8R/RjArwHMJqI+IrqCiD5HRJ/Lv+RxAK8BeBXA9wBcFcV5wxAXuYZq4457nDvnsEh9yG5picfE32NF9kq8hekQIGDyTKwYubIkDx+wjOghLSnpcQuP7+wG7jge6Gq1/t7ZbTym1nTxsd8ZzkrdNrJd4pgi7RWw7iEvtw776plqE4nxF0JcKoQ4TAiREkLMEELcI4T4jhDiO/nnhRDi80KIo4UQJwghqp7AX4s/vqgD1LK4x/pt/bhobnukPmQ77pBKWBIPG3MLcNrBr+OvR+5Hz0c34RcTTpe+b3I6hZvOPw6pZPEqO5Uk3HT+ceNqn0N7AAhgaA+G138eXbfc5HltOjvaMWlC6cZXtuPwsxu07yGvnWW5fPX1lsTAlI+GrfCNullI1DK87uOdfsz0oiKqKCo5VavTx3a+iZbmaG8NXeBWkSgEIo/v6Y5StU87c+isDR8BoL82Xo3X7fO1NCdxYEQvv2BjG3BVTYSJ+ywoXO3L+IGbuURA1I1eZMeT9YQFxitAg0w8pk1LoqiIPXLFY8rnVJ+NgEJuvpue3n4sevg4JCTvzAnCh9+/z7PKVdd0Jcivwnk+3XfoVbUbFK72ZQDzCl8WdguIc3v95e4dxml7Jtty2YpcZYz6BzO4Zt32QCmrpvGNcksKJ3yKzNmGdSA3Vfr8gLAet1fwqmth2njdBLfL0B3rcB63XGnFjZbEUHECxJfiDBv/ALh95arqVvePzrS2IOyP1dRYmzR5iWpMqsAtIL9+uviLPTneNroEw6K56Llh0YzbRsczfXSNXdZs2l0S3wi6D5btjOyaC9lx/U6oJosGTmIoI5L4Eh65uqYnADb+ATCR4AXGf3R+G3Wofqy6blJuTIy1KutFRlgDIgvc6nAqgrqxP9vG3AKsyF6Jvtw05AShLzcNK7KlmUP2Nb6x5wVc69olrd/Wj2ULZxeK4rwa3shob00Hii2YTqimi4ZaTGKoGTy6ydUibPwDYPKjtX90zh+u6fFUP+LLTzvc2DiZGmtV1ov73GENSGdHO9Z84sSiVbaOZ15WV3g7P9vG3AIsGLkLH37/Pvxj9hvSlFHAMphu3SOgdPL1sxuysesDTMZr8rgb02pgrvYtIx7d5GqRhs32CYOqQCdJhJwQRUHX+auf9t2owysTyUS+Waf/7j6u12TmNDRehkSX9eTObtF9Dt2YVF3VdPLIuk5iznPJrv3px0wvSF+3tqRwMDuGTDZXeM87w1kse3BH0fud16J/MFMSS/AzofrZOdRTtW+sGtl7dJOrRdj4B0BnfNw3Z9BGHbofse6Yh7RYufFAqWQxgJJUwGvWbUeCAK+kL5O0Qb+phjoZat2qOIg8sm4Clk2+OiPTserJIuMPWD2Nb35kV+F9N/a8ULTTEAie7dOI1cCRpa3u7NZKfRhz5krLx+90/bg0o2oNNv4B8FMjoPrhAsFT/pR9fNMp9K48W/nDmdCUkBpBTbFqEV7aR371knQy1F6rYp3EhD0WncaQDQG+XVqqnsb24z29/VIXk234/aZdmvaPjtVKOSSRaG/ZQVrbYNtBWsD/BKDpJlersPH3QPWDMt1e+9klmJ5bdcyuRdaKX/XDMQlSe6HbdQQJbN7SeYJUEC5sRbHb/XLg/dGS1xGAy087PPJOY2s27VZmDQUVbrOPqxpTvRV4RZK2qgvSBjHac/SaUbUGG38NUfygglYSm5zbr2KkKelUEhOaElJFTZ2rIah7Ioyf2stQy4qtgHH3mJeIm+y6q3oaE7wL58rVO7jeVGojcXXVYZA2Stj4a4jqBxXEuHmdW3dM1Q+nJZVAJpvzzGXPZMcwMZUo+MoXJTZjeVM32mg/DtKhwE75ymnZwtklvX5TCSpLqmFPb3+JAqlqgpTteISwnrt23faiScPkO+9adJy0p7HXdQ3iYjKl3gq8TF1dWuowSBslbPw1VPMHFeTcuuwSW3/etIhpcDiLOy4+CdsfW4vl2XEJ5pbMm3q/qTuV309xggIvnSMnbkOt605mr96dk4ap3s/Fp8wsZAAlNJlETlqak7h23Xas2bQ7cn98vQWFI9HeqsMgbZRwnr+GalZM+j23u57Azi4BrCDjByY2ITtmXr/ali9c6pq0vqTtoqq4Zc2m3SXnyI6J8Xz0AOXxsgKn+557Qxu/cBpwvxIWugI79xhOP2Y6Xl99LnIaw0+w3ESpJOHAyJi2SCuMImc9FnjZFdLOrnS+mLMEOP8uYPJMIC8fjvPv8ue3rzNJByds/DVU8wfl99xOd8WixGZsbr4av5twGZ6b+CXceewrygwV+7hOUknCgfdHcdSKx5Dz4TfV7lYClsf70TmycRpwP0Vb/YMZHHh/tKQSWab3IwDc99wbyl7CNq0tVnzAPSm6i7TCthWtZoFXrGWk5ywBrn0R6Bq0/pYZfpWBr0NJByfs9tEQteyzF273xkVz2wuuBdvdIfNTA+OGd1Fic1GnrEOxF5O33YhFCXnDFLcqaGtLCu8dHC24RAZyUzEjIem7K/Gbal0PT10fKPPCr4tNJrAGFH+HwyOjyslwMJNFKkE4pCWFweGsNlVX5I+rq1fQTbrOzxZFfKkaBV41n2WkSweNOlsoZrDx90D1gyqHfr/7R7R+W39h9WbaFnB5U3eJmyZNI1je1I2NI8XG3zaUTgPpNnSytosqv6k2SPdwsMwLlfGVrcZb0yl0LSrO4JF9T0BpIZiTbE6gpbkJvSvPBmBWiRxEEG6yQ1NJF2uQtZZUUelc/5rPMtIZ+DrPFmK3TwDCbtFleOm3mLYFbJM0RweANtpf8ph7YpEZOKd4mpffVOt6UGVYeGReeOkc2ed54O/6sP2D16Hz4eMKW3fV92R/dp2+kNMYL1s4Wxm3bm1JFY7plwMOTSCd68j03urp7ceyB3YUfd5lD+woqxum5rOMdAY+4D1bK7DxD4Cp0JYfvH5EXlkogGXQ/kTT5ccRxdr3TiVKL5XSjbkFuLjle3q/aR5lkO7MldaOwYlB5oVqQrml84Tx83x8Hzp2rCzyzWY2fB5PP/BN6ff05W5Lh0en4unuuHX5aYeXTADpVBJCIHDxnDMYrotNmN5bum5p5aLmZaR1Bj7gPVsrsPEPQDlWO14/It2PybmiPXTxV0tu2IxL454AnH7M+CQRVH/IFyEyL7yyPoafWImmsYPFY8YIbmr6ITY3X43XJlyGzc1XY1FiMwCrf4C9mjYKrO/sxi2vX4rXJl6O5yZ+CRckNhcmoSFJsZcf3D19vV6nQ1Z4pns8Cmo+y0hn4KPIFoox7PMPQDlyqr2KWmTPOyn4WVeUapA8Mvlf8Mh/H1N4rQCwfls/5h0xBQA889QnNEW0RihTefzEzFvSx6fQeyB6DwAwg/ZhdepuIGvtZOzrZevsKP3kjoAgwQqgf33SfwEf7wDmnKHVDDKJA7h3GCa9f8MQdUwgkqSIqMTXghzXS7OnziQdnHAP3wBE3bPXeVwvqQKVcQDUPW9VActDWlI4mM3hrLFf5Ct492FATMNto0tKMoOi+Hzlom/l0fKMJNlrc9OwYOQuAPoewQXuOF5RJToTuPZFX/2WZbLO7msa5t469t+fwLBLbRSwKru/unhOSUW0n2OXDXe2DWCtvMOusMt13BqAe/iGRJe7XK6calu0ra01jYHBDNZs2l1yXlM/tROVy+Cd4SzOGvsFVqfuxozEPiQImJGwVsi2i8Sm3H18w3B386dK2jmq1jTttK/gAjJaTXtkfMjuBdVyylb11N0zYe6tCYqYARHhhg0vSNNOq/69lqtDVh123ooadvtIkKVVLntwB7o27sJQJltYlfuV5g1yXnfOdE9vP4ZHShUqdX5WXa66LDW0RZEaGtcMjpPOXYqVD43iGvETtNF+DIipaKGDmIL3Sl5LZLmAbk3djRePPRKAx3eo0Ifpy03FxaufLlJ5tVHttFRyzrIdn+m95XyvatI5MKIPSFf1ey1XOmWdp2lGQSQrfyL6GBHtJqJXiWiF5PnPENFeItqe/3NlFOctF7Lsl+yYwGAmG1lqp+l5nSsze3Jwr+Ba0ylcNNfyF8t2KrpMEj+poZXI4PBVLZqvzOx8+DismrQedzd/Cke/fx/Ob/pPrKF/xftCXdmbphGc/LtveJ9TEhC0m8TPffenOLnnHyBclaF+gqCyvsJ+UjudqaxBqVhmjqyStlzplHWephkFoVf+RJQE8C0AZwHoA7CFiDYKIX7reuk6IcQXwp6vEpishDLZsaLOTeU8r/24LiXTKXTm3jHoirgGxDTMkEwA7tTQSmRw+KoWdfl0WzJvoiv1XXRddlzep3s2cOuPgMzb6hMO9Xmf0xEQzA31YSA3tZA5VVT85qgM7eywnpfFb5wr9daWlNYV43VveaXoAnp5bvv5imTmqCppT7wM2HF/9OJrQUTdyhV4jilRuH1OAfCqEOI1ACCinwC4AIDb+NcMOjeJk3eGs76qL4Oe116Z6RQq3bgNiD0JHLnisaLXySp47ZWtrCdxOXHrE9lB6D89PB1IfrX4h2hSep95R3/CyTPMKlTzGR9HO7T6NzdfrRa8m7NEWhnunmhMpR+CvIYAz4pmWUV0ZLgN6cgB+ff1ypNWEDZqo+u381aYrl81OmlEYfzbATidon0ATpW87iIi+gcA/w3gWiGEJIUiHui0WtxEWcYuS+d0iqyZSgfbuI1DT29/ScbJxtwCIIuCXv+AmOrI9hGBW00GQadPVPJDNPHpqvTcgcIqcOB+85oN5+SscpfpfMomK3XnuUxe4ye2UDHZB5khVTHU5z+d0tTY+jmu12JCdc4oW0VWmCh8/rLKd7eFegTAkUKIOQB+BuAH0gMRLSWirUS0de/evREMLRidHe3GPtQog2WdHe24aG47kmRd0gQVxxpkhj+dSuKQllTJ40CpAVG1F9yYW4AFI3fhw+/fhwUjdxWleZYrvuHE9rnbY5MFoUsyNUx8urICHgBITymk/PmpUHX68gfENO/zu/BzrziL8FT4iS0ElUcOpNgpM6Qq/Prgy6W0qVtM6M5Zw1lFURj/PgAzHf+fAWDA+QIhxH4hxPv5/34PwFzZgYQQa4UQ84QQ86ZP9775y4lO98VJlMGynt5+rN/WXzDyqsbqlP9jpwHedP5xRkYg6EQlTQcMo3PueO/wrcdg80PfLlrBGq2qTUrvZRWai78HXP96YVXm14DaaZhrRpcggwn687vwc6+s39bvaWjLLeMcWMPKNKNGdb1091a5jK1uMVGn4m9RuH22AJhFREcB6AdwCYDLnC8gosOEEG/m/7sIwEsRnLeseFXUAtEHy0zdAgLA7yXFSV7bepWbIGngTiqaOML6R12B2lW0FiOJXGHHoQpCA8IyBs5tvkkFp4cWEWDuEhn35Z8B7Ozw5etVufVGx0o7rDnjDrriP/f47Uk6igkgsGKnyt2WngI0T9JfL697K6yxVblvdAHiDUvV56zhVpGRVPgS0ccB3AkgCeD7Qoj/IKJVALYKITYS0ddgGf1RAG8D+DchxMu6Y8ahwlfWPtCprx+FjLPz+CZBZhuZ8Tc5n6x69KK57cq2iDZFfmSPqlctivc6K2/dPv8SarhSU2bIVfElAnDHxSdpK37LVW0OoCQ5wIn2/gtTXau6twoThyJ+YHLveY1LNTHo7nfVpFHF+9O0wjeSIi8hxOMAHnc9ttLx7xsA3BDFuSpJOZtjyFIMTfVgVD5+L3Sr3HlHTFH2/y3Z4YRZfSle46wr2JhbgCmpZnRNWi//0UXZUKPCmRqylboq5bOtNe25+i6nnr5qR2jHpJT4zbRxorqHMm+r03aTzWapoV5BXdUuUbcrCPNZqwxX+FaJIO0JActNcNP5xwU+r2pCcz7uKf4VZqureK+zriCdSuKkc5cCHTdbfl/ZlYnCp1qFTA3ZpJ9KEFJJKmr1aE+4167bLj2Ol9R3FIkIKlegUcZZUEE0XYaWClPvhXLR4nG+OhV/Y+NfJXQ/TmcLwahdTSaoctTtCeHTH7gINya/UyyjbFqYI1lFjSYn4u6mT4FGUPoZo/Spmuael7FNn7R6PCfQmk5h0oSmku/ZS+WzHAqzNu2aVNKyIVtle5HLmn1nyomFrHtD9/4aNfA62PhXCZ2P39lCMA64V6v3vncK3msexapJ69GSecvfVleyimo6cyW65ixBl+z1QSo1ZfjNPS8Tqkl/KJPF9ptKv3NZkNjZj8FLClyH1w4vzLEDI1tljxzQV2oDZt9ZIXjr3imIuunL6wc2/lVi2cLZuMZjS++k0r1ZnchWqw+O/B1+3XImnu0KIG7nZxUVlU+1nLnnPpicTkkrsu2Vuux7vmhuO+577o2CyXL2Ywiqp28ipeGWBUkSFaX9+r7/ghZnyQK1bky+szlLgA2flT9XA6mZUcPGv0p0drSja+MurSGwubHnhaIfv1bzpgxUvU9rFFvusLnnAZBlix2QKLKmEoRlC2crDfKEpoQ0FfSadduxZtPuQAqzpoHizo52tO95FG3bbsNhyPd7eHcJbtgwUnhe9XlVTXEA+IuvFC0A9qCkM4Kf72zyzJpNzYwa1vOvIl2LvIuzenr7iwy/TSV12GuqT6uqQEj1405PKUubPlmB1H3PvVEU1LX5wMQmbeaOrg1j0Aps4wl9ZzeOf/7f0U7F/R7OGvtF0f3nWRAWtjhrzhIrlbNrCFi8Nvj+6bt2AAAgAElEQVR3FrYvb5jixphR9yv/crpLwh7bZMuukmQAKrfyrorvNwi61aUqdnDOrWXx9frJ5hrMp3mGrcCW3Xuqe9Q4UPzUKqTxftFDdr+Hvx8clwHx3ElEWQkbZicYxo1Ywzo+Mura+PuSCK7Ssb1qCXQGYXI6WL4/4G/iiqRPayXQrS7tAqAK5WP7MeRemTt2u01dEZ4qTqS6R71EBAvfsaYuwzlReO4k4lQJ62fycMYpKAEI13dQ5uywclLXbh+v5ihxPbYTnWvlwMhoIMG1IJotnR3tePbj+/D6X16PZw8uRufPF8Zvy+u1uiy4Dgatv8v4g1V9b+7yKOcOatnC2Ugli19h13XYGj5+zue1GnfqAh3SkgIEShoWDacPlZ7vTUwt2vl5ugZnKbLXVI8HJUq3jFvQzW34bWo0WFzXxr+cgcpKBUF16o7ZMRFoslEZhZsf2YWTbn4SR654DEeueAwdq54cnxDKpaboIpCKpP2DVzlWfKwuA51fgkow7vLTDtcLsUmyEIFxVc47Lz7JWIjO6x51Kn22NDch61ISzGTHcFv24hIfeQYTMDB3eUlaqHtcn2j+FX5KV1mGeNu90rHglSfljwch6nvUNEOsRoPFde32KWcBTDmP7eSZl/XS1kEmG11Dd/f/lz24AwDQ+XOD5ikhCeRK80oD1ATzenr7cfMjuwqfuyWVQHZMFIxgGDdhEFfZmk27SwxwNidKmvKYHld1jyaIil07He3Ke+IH751idUfLuz6G04fituzF+MGvZqDtt6U9jMcLAf8fbhR3oymTLwSsxKrZpMGPE6/UU5OxRZgdVmnq2viXM1BZqSCol3EPMtn4EZGzdxedB8svXeurk5eNbnVmC29J3tfT249lD+4oyr4ZzuZKXhdGJ8evNpTpbtL0uCplWluewTm5aRczc84F5izxnJyLxnXH9cDQwZLjlZA+RP+8H+0lnXxDV6t3E5YNS4E3ngPOu916TBWnoCQgcjWl4yOjrt0+5dQ7L7eWuo3OuNv54X7RNXSXMTCYqUhDbHcnrxkJK72w0MlLtn1XTj6k9euv2bRbmnapG1cReVeT6GrFW11/hS995YZQbiIg+pRa+x5t1SQG2JObfU8sSmzG5uar8dqEy/DshKtx57GvFF7rK85luih4/89qt4xfN472XjRowgIBbP3++PFVaaEXfqcicaNyU9crf8D/6svOgrErGseEupWhybGDpIO6m3ynElTiDgAg76FmgMx1cOD9UWU+eVtrOjqZBRU7u/HriV/Bh8Re5JBAE7lW4bLt+85ueQYGAKn2v4Mg2ThF581fC4I1OX0tdTdWvAtcuy6DrX94G7d0nmB8fJty7Cbt+gFdrcDAYKZQzHX88/cUUjvbsQ/tL9wEHHkIMGeJvziXqUCbTpdH5cbZ8FnrOfd3a6IL5NWExSn1UMOKnSbU9crfL84sGKB0e+x3VRckq8b9nneGswABMhXdoAFfoLSt33knHiZ9XdLeXci6YkWlWZ43podiLxKEUsNv4/zB2gZY5UsGtCtF09W01PhKjJKd+y4A3PfcG4F2AOXaTZq6Dk/+3TdKcvqdhVjua2bvEn438fKSzJotR38RGdFc9Hql+GaQGgDZd+u+R5Xv7dPvEpznrWCGWKWp+5W/H3SdtIL4foNorUtVHzXuiaiyi5pefBCbm3+ENsqX8OebuP/FhKbxsZZL2TBIVoXpexQBv2ULZ5f4/AGrb/IHJ6YwlMmqd2oePQkEUJY4QdCiQl2Mp2hy8zDCzp1JScMdV8HTNb+dhbnZK/Nxm/0YEFORxkFMpfdKj+/8Xh+9zsoM0k3qNrLv1nmPKpuwzNCIvEE9MRTiD3vyfv8xbVwp7rDxd+BlSP0a2iDpoH7PEUl20c5uLM9+Gy0J64c8g6wSfmSBRzILPN4cAUGyKvwEmiWvtY2mne2zKLEZK5q7cRj2gz44A7hwpRXolGHQkyDqlN8wRYWqwG9rOoWuRccZy2c73YXLh7tLO605jPHAYAb9WICNI+P3j7RDm/N7ffQ6YOs92s9Sgu4+8GrC8sZzlo/fRCfIHSC2J6carvJlt48DL0Pq19CqXt/aklLmkivfk04Z53f75qlVJT9k241REf0e1UqLklC6mPwEmhWv7exoR+/Ks/H7yw7grkn/hTbsA5kEFiWBwGHRjNtGx8cX9XULU1QocyfdefFJ2H7T2cUTh4Huje0unJHYDyl5Yyz7/BtzC3Bb6iq161BVC6BDdx94uSrPu91cJ0i304yigXwV4JW/A13T9iCGVlVC/97B0UJuuXsFpwr6dS2yuneVRWJB48Yoq36PcxstU2rUxRRkq7pks+VYzjmCmyZBab/54Y5AoBjqQ39uasFNBijulZDtIsMWFRolPvgJcHrsElT3caFDGzB+TTYstd6nc/Us/l6whAMvV6XX80X3qIahPdoEgzjCxt+BTL9cl+3j53i6rBpnHMCriCfKhvGF4yp+yG/RVMlRIqKkOEugMAGY+FFVhgoAnrh+vPlHk8EKPEjAMW80aGc3pjyxEndm/hPLc924u/lTOOncpcXfUwSCYJUqKjSO63hkf3kWo/lprkPJ6mTemPQRcFJjLiA2/i6ibtruPt5RKx6Tvs65gitH43itz1jyQx4WzVidXYKflqtvgCrPevLMcRE2L2SGamc3MOo4buZt7x9kUNGxvHFoyX+OGYl96KLvAsnjADjO5XdnISF2yqphjbGf5jpzPzN+zkoaVT9jtKkhoTc2/hUm6AourHy0NvNohXWjvrXhK/iQ2IcB4XBj5IJXuGqJUuLXSRBDG7SGwfRcis+UG9qDgZVHoy2xHwfTh6LlnFWFSlrVdx0rZVWNMfYKUIuhPnUypp1JQ0nL8NsVt5VGdy/aY/T7vhjBxr9COIvHXN5tzxWcs5PXosRmLB/uRlvPfgw/OW4wvPD0Gc9Zgo/cP0kqjVaWvgGmq22/vvKgLhzA/yrW9Fyqzyqs3QIAtGTexOjDX0Tv79/BDVuO0Eoo1AJeAeqTxVS0077SN/rZ+ZUb5T2aH6MqlZQSpXISMYSzfSqAu3jM9m4D3oU8zk5exbIHAi2ZN41VC02kAyrascuko1IQlcagMhRBinmU3cEOKZYVnnV2yWfNCaumwEnT2EHMfH5NRaTCy41usbFm027cml2CYVcRWAYT/FeMl7Ozltc9KnseyO8Iyqd8GxVs/CuAqqtTe2saz644w1Pp0V6NL2/S5FZ7oJIYdu44TF4TGSYVw0Fa/4Vt0+cH2bkSKWDkveIJa8f9wImXFT5rX26a0uXxISFZDaOC/ZKjYGc3fj3xS3htwmXY3Hw1FiU2F55qa01jYDCDjbkFWJG9En25acgJ65qsGLlCPenKjLzJ4iDM5OB1j7qfJ4lelmkaaBXaQ0bi9iGijwH4OoAkgLuFEKtdz08A8EMAcwHsB3CxEOL3UZy7FgiTpud8TZtsmwwY+RhNfMYV9yt7BfDK7cIJmX4pPdfIgfFMI5tsxtKtz7szLl79NNYNfxYzJN/nn2ia9FSx7JcsoyDVkQGouGDwp8l/xLKFswvuz4254iIwZbOaR68rLsayjXxTWh9ziaLtop9U0a5W+Wu8fp9Vag8Z2vgTURLAtwCcBaAPwBYi2iiE+K3jZVcAeEcI8VdEdAmAWwFcHPbctUKYND3newfENKnBkLkfVEFDL0MeK79y0Cwck6yQqH5w7nMZGIBlC2fjzocuwSqxtmgnNyya8XW6FKkkFclOxLJfsgqF7tFXmh/AGRd8oXBvGWcu7ewurcIFrHOoMnHsax1BlpUvgt6vlR5nnijcPqcAeFUI8ZoQYgTATwBc4HrNBQB+kP/3gwDOJJJJlYUnTCcm2Xuj6OwUxp3ifO9to6V+UplLI4igXCw5c6VVuOVm5ED4bXEQl5IJBjGHzo52LLjwKtyWuqrY7ZG9Ej8+eBogrLaK5ZQKLxuKVe6h2FdUq+IpYGe7QTZ8FsoObSrsa12ujDIVQV2OlR5nnijcPu0AnNNdH4BTVa8RQowS0RCAqQCKlrFEtBTAUgA4/PDDfQ8kjP6J7L3LHtgB0LiwWpgm7UAwd4rzvY8MLsCUVDOWp9ahJfOW0lURRFAutsikIE1y970o1w/OMG3U2mHdjPmr/7FkV5jNCbQ0N6F3ZcT9bXWEdYHZx1BJbLsyYDo7lhQXfD11NfBw/tyzzrbiJF459ukpVk2H6lpXuml80KyxKjW3j8L4y1bw7l+syWsghFgLYC0AzJs3z+d0rzZ6X+7OtyL0CKyWqGlKNPSDGtEw7pTi954L4Gbt6yvVX7jsPLWqWKrBicm2WGfQyvWD82kA5r37U6xr7i5RU63odxWFC8xLYtsphNZz1fixZeeWuXlKIOCcW61/qq51uXtQyAhSiFaNcSIa498HYKbj/zMADChe00dETQAmA3BFxcKj+sGMCeG5YvfzYwvzwwxbrGVCxaQAyo3XKlwnCeBl0Mr5g7MNgFO7xm4+Aowbq/QhWNP8LpoxCqA4OLrtg2eFH4cpUfic/VTD5rKWBMecJepKby0EzPvX4qwbGZWShChH4kAF6gOiMP5bAMwioqMA9AO4BMBlrtdsBPBpAL8G8AkATwuhbO0QGJ1uudeK3U9f26BGNIxbyg+xkwIIilc3KFlqnY2JQXNmi6SnWCtJXaqhnx+nbPJ5+PPA2CiAfLOazNtwRzRaaATXp7qxZeEX1MeOmihcYH7dZXZGlN/3+dXPL7ckhKoX8IbP+htrpaUrEEHAVwgxCuALADYBeAlAtxBiFxGtIqJF+ZfdA2AqEb0K4DoAK8KeV4ZXb1rdin3ZwtlIJb1j0GGMaBhZXj9Uqr9wIPzkM6uKaGzEmPpYqkljaM/4D9aZkjmqmfiDFJvJJp+xERQMv4Y22l/Z7yqK/sxB3GV3HO/dwN0mlbaUPePWTUu3c4l5kVckef5CiMcBPO56bKXj3wcBfDKKc+mwfzBf7t5RaMHoRLdi7+xoR9fGXdJep0ki5IQI7aappC8+VimbNn59y/ZjD31O4UumcSPvPpZKe4WS/t0cQdwiIQLHVOZAn5stR38Rxz//78UtHP26wEz657oZ2iPP5rKhJCByxTst2Q4MqF6fXa/vOcZCb3Wn7eM7j9jBkKLJdU4IvL5a0dXJByrXkt3cJTaCXeUiiBG1Hy8xLG6FpPyxbF+yLvDo182hfDy/i/Cjd+9FKo0tR38R1zjuhzuPfQUn/+4bZTFuPb39uGHLEThr7Ip8y8V9yCGBZDYDstNeTd0WgKM5uqFXd2xE/ZzIWXIbNrLFQ89VVoPrMXk7ybJj8j3HVOitLuUdgro9yq1to3JLvTOcrf2cfBO8jKvKJSQrs1cZl8zb1vsmz5Q/P3mmfzeHbiWu2tZ7uaycUAL259pyws345y1HFO6Hue/+FMdvu9Gfy8kHtityY24BbhtdgoNoRhPlrPQ8v+dy6iOlp4QfnPu6yxYPuWzpBFLJzlom33OFd3Km1KXxB8bbzb2++lxP/Rybcmvb2JPSIS0p7etqUcjLCJ1x9fKru4XXVMYdGM+sURXc+C3G0f3AVYZGNmHNu6LUzZFsBi78buFzXfPbWUU71uVN3UgH1HMywelyDKMdFYrUpNLrm2y2ivmcC4EwAehyaecUfc9ASVZ7BVI2g1K3xj8IlQiUdna0o6XZ29tWczn5JuiMrt+KW90PaqhPL8plIirnxH697nyK9/V8dBPmT9yAo/54K+a/eB62nHhL8Xkv+FbRed3fu1bPKQKDVqTqaqod9eh1wM1TgK7J1t+PXlf6nsw75oNomlD8faSnWMV9mbdRtBAwDQ4DxQuNIAF753u9rnFhYTJk3hM4BlAZMi4jYd68eWLr1q3VHkZZOGrFY54eUVvxs+5QpUx2tULuyqFiv6+TW48qFVEDotWEd45XVb2qOJ87tRewdpK6BcX81U8XxYU2N19d0PwvQlXd6tPY9PT249p12yF053J+vkevA7beU/qaeVdYTVdMe94W4fqOVTr5ss+cSBX7/IHS66A6ntd9ImvjGOAaVxoi2iaEmOf1Ol75VwGvGEJN5uSbotLND+JvP+fW8so3P3qdlbNtrxhlhl9zviCpvU7X46LEZqRxUKJwQcDo+5FoE3V2tOPy0w4HwVA7atu98gNt+y9rMt7wWY3hV6RSu79j1U4q87b1Ge36jskzgc5vW7sn3Wo7aB1DufSfYkLdZfvUArIiLDt3JUlUZCDqMutHRpCK26IMkz2WUXD+OMOszlRqkk48injcLpxFic1WRk1mH3DH+HvdVd8XzW1H04sPYnn27lIfPGCNKXtAPqYAmSW3dJ6ACxK/wsznH0BajGAUCSSRA8k+nzKLKiffhdlMninX7JF9x14ZNGJs/H1eVb6643kFYqskuFYpeOVfBWSxhctPOxzpVLJQn1DXWT8y/Prhne+zYwlO/RinTzeIb/ypVdAafrfxkeDc4RV3YRsf45aN3y1RYF2/rd8S75Mafg+CZJbs7MbJL9yEQ7EXREATchhLTkTXgYtw1P2TitVsdVXVSsja5Z13u9l3bJJB47UCd37nIwdKA+0mO8Qoit9iDPv8Y4Lb12tTFt9/FAqOcULn01XtKLwmFmUMwgElgQu/o21ivvmhb+Ma/ATttA8yEfO3MB2nHfx6yeOvTbwcCb9SxkH90Yrr15ebhgUjVqC7EKvo/z9yn7+O9BTg+tf9vcd5jyqvgyIeJPPVJ1LAhL+wAtGm93yd+/zZ7RMTKlb9W6WuQWVFtz0PKlpmUrwjxiwf9xPXS3WBOpPP4rzU3WgaO6g8hN22seASyqt7vpObhKmJ9/Tnp4TlbgG8tYl0KK5fG+0v/LugjbXiduuBbfdan985hihxat0oJ3fFClxVD9A8yd8kVCXBtUrBbp+YULHm6fUYxNJtz4P6bVWNZGTY/QXc7qSnVmkNP2C1bXS7hGYk9uEvEget1aoOp9HVaRN5obh+A2Jq8f/thch5twM3vW1p7TRN8D6+n7RPGX7rMqL01asSFOoANv4xoWLN0+sxiKUzDmH8tmOj5mOQTaBe1zSVxp6/XYbrU6XFVc0YtdwUztz39BSEbhQuQ3L9hkUzbhstNnQlCxFTGecoeiT4iQfVua8+KtjtExMq1jy9Sl2Dyopsez7rbEe+uUsHyCTY99QqmChwFuE29jrXUT4ecTIA8byiuCrzjtxNEbRRuArX9RtOH4qVBy7CxtzfFV4iXYiYnC/qHgkm6DLH6i3eFQI2/jGiIkqcVeoaVFZ2dlt+dzvVMPM28PwPHV3ABAoTgKnGehBDastUFFJPJRtrZ8AwH39RColr3VkGE7gfQ+eYAFqG+rBq0np8oLkJP3jvFPVCRDUOmRpnWAJ+lhL1zyi09+sEzvZpROpp9bOz21J2VLV7dOKn8lcVZAQgVRRNpYETL9P3nnUHZXXn0GWVmGShPHpdaZ1C2GNG8R7ZMbzuxaiybrTfacBjxhCu8GXU1FMQS9fn142f1fyZK+UB12Szpd+y+HulPuhXntT7wJsnmVWeAlaXMRVePnBVgZouLhAkESBobYaNqebOE9dHk6Rgqr3fILDbp1Gpl9W/H4Pu5RaZdbZlwO3//+0/A7seGncnuVfu7uu1Yan3WE20goDxDCLZebzQFahF1cvAJkz7QZM03J3d6sph3y0ga1d7vxyw8W9E4pbrH2YiMm2a4o5ryK6Bs3hpaI/lwtG5Sdxj9hpL+pDic6oMv42qHsHr+9MZsLBxBFNMvlOTCUe3Evc7tjNXWv59HbWc+OATdvs0InHK9Q8jtwuo3TOJ5HhqpMwdYZKmqLomqjHPOlsjS0DjwmQlT2kkE2QG0uv7Uxow8tezIGgigOl3apKSqZvI/I5tzhKPJjOa61OHsPFvROKU6x92IpqzxFJ2dP6oKQHkxiwf++K18riG6Wf1Y3xfebK4sUdRto8msULkNJ3HJAbS6/uTauMQMO9fvXsWRKFFr7o+T1xfrLEkmyzdE45Kw9/+vv1qNp1zq+ZJUZuuz4Cw8W9E4lQEE8VENGeJlQ+/+Ht5gbd8fr5uF2H6Wf0aXzuYblr9ap/Dz8rb6/ubs8TKPLJ3FJS0DP95t5uNxwS3WN6j143/X+X6yrxdvBvYcb81Tl3g+v0/lx4n2Qwcd2GwHaNu9a/rDleHsPFvRKLc4oclyonIzy7izJVQ6svbBDW+qrHozuFn5e31/e3stgyrHVMQY9b/dYbRj/tN9tqt94z/35RsxnrfuwPW5OTeoakyuZo/IM+sMt0xlrsPRI3Axr8RiXKLHxaZzz6RCvZDVK7I95S6BeYsgadWf1DjqxuL7hymKbjuvrHOPgZ2oNWvYfTzHtOJzRQxZk0C7naQyqYu7+i/ay/idP9XEc72aVTCpOhFjVvrWKZ9bIIu20aW0TR5ZrD2fnOWAG8851C2TFruC+f11I0limIi+72yrB+VYdZNSH7cb+WKDW27t9g15ZWBJL2+ZE2AJn0g4nL/V4lQK38imkJEPyWiV/J/S6MzRDRGRNvzfzaGOSdTZzy1qrj/KmD932v7Lmsi7tUExL2SDer+MnGr6MbiXKWHQbVaV2UP2fITsiCpH/dbEJccJbwbtLhTX3Xfj9JtJxqqUCsMYd0+KwA8JYSYBeCp/P9lZIQQJ+X/LAp5Tibu+OmcFSTgazcRdxrfrfdYq3GnO8TkuM5K2vQUsxW5iYvEdi2ogot+U1qlx1BcI7vNoZNU2squUfn1/UyEJp223Mz9F+/vxj1p6dwzOrddAxVqhSGs8b8AwA/y//4BgM6Qx2NqHb95+8pUPsXjgKaJ+L3jfnOv1El7nM7qUVNNfNMJa84SK91URdjaCuVqfabcaOqCpG5Dm55iTYwblsrjJe7jz7ti/P+pSeNprpS0njvv9vHvZt4V8nHP/UzpY7o4iJ/0WKaEsD7/vxRCvAkAQog3iehDitdNJKKtAEYBrBZC9MheRERLASwFgMMPPzzk0JiqELRzlh+UTcQdj3upl4YZp59qWK9VqOkqVVYxq/uMMp+2Sn7CHoP9HpMK8DA+c9uv74yZzP2M/1TUelSorSCexp+IfgbgUMlT/9vHeQ4XQgwQ0YcBPE1ELwghfud+kRBiLYC1gKXq6eP4TFzwm4Wh6vKk6/5ESfkE4HQbeLXgC1Nf4MfoeEk+OCcMpdaQqyeBbYzPv8v6YyqNYTppebm1TM9XJG+d/85s6eTzbg9fd1DnbRbLjafxF0L8k+o5IvojER2WX/UfBuBPimMM5P9+jYh+DqADQInxZ+oApbFTZGGkD5ELd+ncPnM/I28i7nYb6FanOkPopUvjx+jIJgobd26+TmtIpdDpR5XVdNLSTeCmmlDuz2NP1lHrSHHWTmDC+vw3Avh0/t+fBvCw+wVEdAgRTcj/exqA+QB+G/K8TFypRBbGebdbfuOiCtYrvFeSO7uBW4+yMoRkht8rKOokTE4+UJpbHiR33m9g0zS/XeUzt+sJnASpBWgw6eS4EtbnvxpANxFdAeANAJ8EACKaB+BzQogrAfwNgO8SUQ7WZLNaCMHGv16Zs0StnCgzVkHcPoB/t4FX0xfbHRFVzMKvUmnQzmF+MVkpq3YIfuoHoop1MGUj1MpfCLFfCHGmEGJW/u+3849vzRt+CCF+JYQ4QQhxYv5vyX6dqSv8ZGFUSmdI1/QlPWV89R6F1lAQpVK/n7ecgU3VDiGK79X0eabssLwDEz1hc8bLYdh0xtsZcwg7Ge3sBh76nHz38NDn1LUPRrnzeXfa5JlWRfFTq/wpWvpB5tY6c6UlquYk2ey/FoAzcmIBG38mevxop1RKZ8XUeM86W/74lA97F67ZK35tKqpiJ+CVOz95piVP3TVkGc4d9wfvgRAGd89vVQ9w01gHUzW4gTvTGOzsVsci0lMsSWhA0+Tb1bRdps/j1SDcjZ+G8k5U5/F7PL9xiajOy5QV0wbuLOzGNAa2GJs7RTTZbEn8OnPSpUhSLZ+4vth4+jH8QPCgp2lcQmXcd3ZbY3e6u0xSMMvVBKhe+knXGOz2YRqH8263mqw4XSkXfMt6rhCg9YG7OYlXfwA3QYOeJnEJVdD50etKZS1svFIwTc/rp7tW2DaeTGDY+DO1hV/j4kYWyPTMsTc16qL0tam05buPMqhtEiRXpaxuu1f/Wb165no1kfFryOPUT7rBYOPP1A7lWiXqDN7kmVaXKWMVS1G8szjxsnFBtYBBz57efsxf/TSOWvEY5q9+Gj1j872D5DrFTx263YhXcD6IIQ/iSgq7AGAAsM+fqSXKJRqnlHpwBDIPP63YLz1yQO46cb5HJnHgFF5T4fCBv5/6IP5hJIdF+DMGmqfhtneX4IYNI8Di+ejUBVlVn0mli2Q9Od71TDVGXZFYEEPuRyQPMBOdY4zglT9TO5Qr4GjiRnG7i0z6wAZZCbt2NxOyQ5hCf0aCgBmJfViduhtnjf0CazbtDvaZ5n5Gs4txCcf5XVEHqZHwW+fBbqLIYOPP1A7lqgYOUmtg8p4gk5VH/KGFRrC8qRsDg/nXqFwgqvGdd7vlivKKYwQxqEEK9vxe+3ItABoQdvswtYNKc2bW2fkc9BCpgkHUIb3e49elARgZsTbaj7bWtGXoH/78eBvMoT3W/51jk43vlSehbV7vYyxFBJVYNr32O7utJjEytxXLRfiGjT9TO8iMy6yzrWrXOPqA/ej+235+A6P8JqZi2cLZwBNXyPsfP3F9NCJy5RKOC4KueprlIgLBxp+pLdzG5Y7jy985zMZvMZLpStgdxNSQwQQMzF2Ozo524GFJwBmQB6KdmBSkxc2gqtxhlGS5iICwz5+pbSrlAw6aZmoHihevtf4v64mr8/Onp+SbwFv+8PTib+LkRf8z3GeR+eYTqaLzxM6gKlNXc/EaZw3BK/8Q9PT2Y82m3RgYzKCtNY1lC2dbKzKmcgTxqwchTJqpV3qicqKicc0hGQjoIQ8AAAW6SURBVOkpii5oU/TjqcX2h5X6nhsIXvkHpKe3HzdseAH9gxkIAP2DGdyw4QX09PZXe2iNRbUloU12GF7piUGzmM651VqxO0mkrMe9MO1EFhdmnQ1p9XScXFM1Bhv/gKzZtBuZbHHwKZMd886/ZqKl2pLQJitPr4kj6AQ2ZwnQ+e3iz9757fgbcr/s7LaC+kXBcLJSVuvts1YQdvsEpJBnbfg4U0Yq0cTbT+aOGy+XRRg3TCM0MJfGREQ+ZZUJChv/gLS1ptEvMfRtraYaMExNEcZAm0wc1TbicZZV5sKussDGPyDLFs7GDRteKHL9pFNJK/+aqU+CGui4B1jjrpfDwd6ywMY/IHZWD2f7MEZUe2Wvo1yCeVERxuXGKGHjH4LOjnY29kztE3e3Stx3TjUKG3+GCUOcfeUm1IpeTpx3TjUKG3+GCUrcfeVeBNHLqfXJjikQKs+fiD5JRLuIKEdEym7xRPQxItpNRK8S0Yow52SY2FDr2vJ+9XK4325dEbbI60UAiwH8UvUCIkoC+BaAcwAcC+BSIjo25HkZpvrE3VfuhV+9nFqf7JgiQhl/IcRLQgivktZTALwqhHhNCDEC4CcALghzXoaJBeVqLlMp/I6/1ic7pohKyDu0A3Am6fblHyuBiJYS0VYi2rp3794KDI1hQlBOXaFKNCn3O/5an+yYIjyNPxH9jIhelPwxXb3L+sVJO1YIIdYKIeYJIeZNnz7d8PAMUyV0ukJhjHelfOt+dZEqJaLHVATPbB8hxD+FPEcfgJmO/88AMBDymAwTD2QpiGGzgCpZdOUnhZLz7euKSqR6bgEwi4iOAtAP4BIAl1XgvAxTHcIa7zj71jnfvm4Im+p5IRH1AfgIgMeIaFP+8TYiehwAhBCjAL4AYBOAlwB0CyF2hRs2w8SYsMabfetMBQib7fOQEGKGEGKCEOIvhRAL848PCCE+7njd40KIvxZCHC2E+I+wg2aYWBPWePvxrVciMMzUJdzMhWGiJmxg1DQQy0VXTAhY3oFhoiaKwKiJbz3uapxMrGHjzzDloBKB0TgHhpnYw24fhqlVODDMhICNP8PUKlx0xYSAjT/D1Cp+K3QZxgH7/BmmluGiKyYgvPJnGIZpQNj4MwzDNCBs/BmGYRoQNv4MwzANCBt/hmGYBoSNP8MwTAPCxp9hGKYBYePPMAzTgJAQ0na6VYeI9gL4Q7XHUSGmAdhX7UHEHL5Gevj6eNMo1+gIIYRnE/TYGv9Ggoi2CiHmVXsccYavkR6+Pt7wNSqG3T4MwzANCBt/hmGYBoSNfzxYW+0B1AB8jfTw9fGGr5ED9vkzDMM0ILzyZxiGaUDY+McEIvokEe0iohwRcUZCHiL6GBHtJqJXiWhFtccTN4jo+0T0JyJ6sdpjiStENJOIniGil/K/sS9Ve0xxgI1/fHgRwGIAv6z2QOICESUBfAvAOQCOBXApER1b3VHFjnsBfKzag4g5owC+LIT4GwCnAfg830ds/GODEOIlIcTuao8jZpwC4FUhxGtCiBEAPwFwQZXHFCuEEL8E8Ha1xxFnhBBvCiGez//7zwBeAtBe3VFVHzb+TJxpB7DH8f8+8I+WCQERHQmgA8BvqjuS6sM9fCsIEf0MwKGSp/63EOLhSo+nBiDJY5yexgSCiD4AYD2Aa4QQ71Z7PNWGjX8FEUL8U7XHUGP0AZjp+P8MAANVGgtTwxBRCpbhv08IsaHa44kD7PZh4swWALOI6CgiagZwCYCNVR4TU2MQEQG4B8BLQojbqz2euMDGPyYQ0YVE1AfgIwAeI6JN1R5TtRFCjAL4AoBNsIJ03UKIXdUdVbwgoh8D+DWA2UTUR0RXVHtMMWQ+gP8B4Awi2p7/8/FqD6racIUvwzBMA8Irf4ZhmAaEjT/DMEwDwsafYRimAWHjzzAM04Cw8WcYhmlA2PgzDMM0IGz8GYZhGhA2/gzDMA3I/wd2/CkjS78k6wAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1a3fdc20e48>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "X, y = datasets.make_moons(n_samples=500, noise=0.3, random_state=666)\n",
    "\n",
    "plt.scatter(X[y==0, 0], X[y==0, 1])\n",
    "plt.scatter(X[y==1, 0], X[y==1, 1])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 因为AdaBoosting没有OOB, 所以需要测试集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=666)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "AdaBoostClassifier(algorithm='SAMME.R',\n",
       "          base_estimator=DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=2,\n",
       "            max_features=None, max_leaf_nodes=None,\n",
       "            min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n",
       "            splitter='best'),\n",
       "          learning_rate=1.0, n_estimators=500, random_state=None)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ada_clf = AdaBoostClassifier(base_estimator=DecisionTreeClassifier(max_depth=2),\n",
    "                            n_estimators=500)\n",
    "\n",
    "ada_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.856"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ada_clf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gradient Boosting\n",
    "训练一个模型m1, 产生错误e1\n",
    "\n",
    "针对e1训练第二个模型m2, 产生错误e2\n",
    "\n",
    "针对e2训练第三个模型m3, 产生错误e3...\n",
    "\n",
    "最终预测结果: m1 + m2 + m3 + ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import GradientBoostingClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GradientBoostingClassifier(criterion='friedman_mse', init=None,\n",
       "              learning_rate=0.1, loss='deviance', max_depth=2,\n",
       "              max_features=None, max_leaf_nodes=None,\n",
       "              min_impurity_decrease=0.0, min_impurity_split=None,\n",
       "              min_samples_leaf=1, min_samples_split=2,\n",
       "              min_weight_fraction_leaf=0.0, n_estimators=30,\n",
       "              presort='auto', random_state=None, subsample=1.0, verbose=0,\n",
       "              warm_start=False)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gb_clf = GradientBoostingClassifier(max_depth=2,  # 这里面也是决策树\n",
    "                                   n_estimators=30)\n",
    "gb_clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.904"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gb_clf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Boosting 解决回归问题"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.ensemble import AdaBoostRegressor\n",
    "from sklearn.ensemble import GradientBoostingRegressor"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Stacking (有点像神经网络)\n",
    "数据进多个子模型, 子模型结果不去voting, 而是再进入一个模型blending, 给出一个结果\n",
    "\n",
    "太复杂容易过拟合\n",
    "\n",
    "#### sklearn 中没有stacking的封装"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
